Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable definition of multiple merge windows per node [JIRA: RIAK-2730] #111

Open
wants to merge 2 commits into
base: develop-3.0
Choose a base branch
from

Conversation

cdahlqvist
Copy link

In order to limit the potential impact of merging, it is often recommended to use non-overlapping, staggered merge windows. This currently means relying on a single merge window per node, leading to increased disk usage between windows due to the lack of merging activity.

This PR makes it possible to define a list of merge windows for a node, allowing the creation of multiple shorter merge windows for each node, potentially reducing the time between merges.

The list of merge windows can be specified as follows in the app.config file:

{merge_window, [{1, 1},{7, 7},{13, 13},{19, 19}]},

@joedevivo
Copy link
Contributor

I've recently moved the bitcask cuttlefish schema into the bitcask repo in PR #112. After these are both merged, we'll need to revisit the merge_window section of the schema to make sure this can be set via cuttlefish.

@slfritchie
Copy link
Contributor

The Riak 2.0 code freeze date is approaching. @cdahlqvist What work is required for cuttlefish-style configuration?

@joedevivo
Copy link
Contributor

This PR needs to be rebased before that work can be done. A list of tuples is a weird thing to map. Here are some ideas of how it could be done:

start/end pairs

merge_window.1=1
merge_window.7=7
merge_window.13=19
merge_window.19=19

but I don't like that so much.

$named windows

merge_window.midnight.start = 1
merge_window.midnight.end = 1
merge_window.morning.start=7
merge_window.morning.end=7
merge_window.noon.start = 13
merge_window.noon.end = 13
merge_window.evening.start = 19
merge_window.evening.end = 19

We already have merge_window.start and merge_window.end, so this would fit in there. We could also just do:

Parsed list of tuples

merge_windows = [{1,1}, {7,7}, {13,13}, {19,19}]

but then I'll have to teach cuttlefish how to parse a string. not a terrible idea, but work that would need doing.

Do nothing

We could require that this be set in the advanced.config file.

@evanmcc
Copy link
Contributor

evanmcc commented Dec 12, 2013

i vote for the last one. not hard to walk people through.

@joedevivo
Copy link
Contributor

Did this even get reviewed, or is the entire conversation just about configuration?

@evanmcc
Copy link
Contributor

evanmcc commented Dec 13, 2013

I reviewed but didn't test. Need to take another look and test it. But I
think that we're just talking about config, here.

On Fri, Dec 13, 2013 at 1:37 PM, Joe DeVivo [email protected]:

Did this even get reviewed, or is the entire conversation just about
configuration?


Reply to this email directly or view it on GitHubhttps://github.com//pull/111#issuecomment-30545648
.

@joedevivo
Copy link
Contributor

Don't let config slow this down. get it merged and assume advanced.config for now

@slfritchie
Copy link
Contributor

Hrm. Please note that this PR, #127, has just been reviewed & merged. It isn't clear to me how much more work/change that this ticket requires, sorry, I don't have time this week to review the details.

@slfritchie
Copy link
Contributor

Ping: Friday's deadline is Friday.

@evanmcc
Copy link
Contributor

evanmcc commented Feb 14, 2014

So, Jon called "no more features" before I could get this in. I am going to tweak it a little bit and add a test, then check it into the first version past the 2.0 tag for bitcask, so that it makes it into the next possible release past the initial 2.0 release.

I feel terrible, sorry Christian.

@slfritchie slfritchie added this to the 2.0.1 milestone Mar 24, 2014
@evanmcc evanmcc modified the milestones: 2.1, 2.0.1 May 12, 2014
@angrycub
Copy link

angrycub commented Aug 4, 2016

create jira issue

@Basho-JIRA Basho-JIRA changed the title Enable definition of multiple merge windows per node Enable definition of multiple merge windows per node [JIRA: RIAK-2730] Aug 4, 2016
@martincox martincox changed the base branch from develop to develop-3.0 January 26, 2022 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants